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Abstract 

The impetus for this study is grounded in a strategic decision by management to measure 
readiness for university education as part of an early alert and referral system. The 
motivation for this project is also rooted in literature that points out that the South African 
higher education system faces challenges with students entering the system underprepared. 
Data at entry to the university, specifically related to the individual student, is used initially 
to profile the students. This profile is used to identify students who could be at risk of 
failing. These students are referred to a Faculty Student Advisor (FSA) for support to 
address their needs. 

Using a survey, 966 students were identified as being at risk at the beginning of 
the 2013 academic year. After additional criteria were applied to our prediction model, 
200 students were selected for academic development workshops or individual sessions 
provided as intervention in the first semester. An outcomes assessment method was used to 
determine whether the number of sessions that at-risk students attend has had an influence 
on their academic achievement in the first semester. The assumption is that students who 
made more use of the intervention services (attended more sessions) were more likely to 
be successful than students who defaulted on the intervention or attended fewer sessions 
with the FSAs. 

A cross-tabulation showed a significant association on the Pearson s Chi-square statistic 
(13.60, df(4), p = 0.009), which implies that students who attend more sessions with the 
FSA are more likely to be academically successful in their first semester. 
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Background 

In 2010, a decision was made to provide a number of interventions to support students 
to succeed at the university. The aim was to develop an early alert and referral system to 
identify students who required support. One of the strategic goals of the university is to 
increase the access, throughput and diversity of students. Bringing the three concepts together 
results in a focus on the success of our diverse group of students. Unpublished cohort 
research on student success done at our institution over a number of years indicates that 
first-year students are especially at risk of failure and eventual withdrawal, which adversely 
impacts on the institution’s success indicators. 

The challenges faced by our university are not experienced in isolation. Literature 
(Van Zyl, 2013; Scott, 2009) shows that the South African higher education system as a 
whole is facing similar challenges. Since the publication of the White Paper on the higher 
education landscape (DoE, 1997) an updated report by the Council on Higher Education 
(CHE, 2014) shows that the key indicators of the success of the sector, such as participation 
and retention rates, are lower than anticipated. The gross participation rate in South Africa 
has, for instance, plateaued at about 17% (CHE, 2014). Graduation rates disaggregated 
by race show that, although African students have made some improvements in gross 
participation rates (improvement from 9% to 14% between 1996 and 2012), there is still 
a large disparity between their participation rates and their proportional population size. 
Proportionately, white and Indian students have much higher participation rates in the 
higher education system (CHE, 2009; 2013; 2014). Given that the number of students who 
gain access to the university is, relatively speaking, very low, it is of concern that only 51% 
of the students who gain access to three-year degree programmes at universities tend to 
graduate after period of six years (CHE, 2009; 2014; Bunting et ai., 2010). In addition, these 
graduation rates are also highly skewed among racial groups (CHE, 2013). 

In order to increase students’ chances of success, higher education institutions need to 
make a concerted effort. Success, in this context, is defined as the ability of an institution 
to retain a student from admission through to graduation (Berger & Lyon, 2005). Research 
suggests that evaluating success only towards the end of a programme is insufficient and that 
students should be assessed and monitored from the very first day they enter the university 
(arguably even before then) and at strategic points along the cycle of the academic 
programme (Van der Merwe & Pina, 2008; Rassen et al., 2013; Van Zyl, Gravett & De 
Bruin, 2012;Van Zyl, 2013). Practices are constantly being put in place in the hope that 
they will influence student success. 

The thinking underlying this study is that improvement of students’ overall experience 
inside and outside the classroom, especially at first-year level, is a prerequisite for ensuring 
that students succeed in subsequent years of study. Scott (2009) writes that the first year 
is regarded as the academic year in which students’ success is highly influenced by their 
experiences. They have to adjust to the new institutional environment and manage 
increased levels of stress (Tinto, 1993; Bean & Eaton, 2000; Upcraft, Gardner & Barefoot, 
2005;Jones et ah, 2008; Hawkins & Larabee, 2009). 
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Literature review 

Internationally, institutions have approached some of the problems discussed above with 
an orchestrated approach by implementing an early alert, monitoring and referral system 
(Rassen et ai., 2013;Tinto, 2013). Early alert refers to the identification of a student who is 
potentially at risk of being unsuccessful at a university, either academically or personally (Beck 
& Davidson, 2001; Seidman, 2005). Such a system is heavily focused during the first academic 
year, as numerous research points to the first-year learning experience as critical to student 
success and persistence (Reason,Terenzini & Domingo, 2006; Pascarella &Terenzini, 2005). 

According to Wend (2006), the student learning experience can be defined as the 
variety of experiences within the sphere of the university that influence learning. The 
student learning experience is therefore all-embracing and includes matters such as 
curricula; methods of teaching, learning and assessment; the learning environment and 
resources; student progress and achievement; and academic and pastoral support. The first- 
year experience is not only influenced by the university environment in which students 
go to class, socialise in cafeterias, participate in sport or learn in small groups in the library, 
but is also highly influenced by students’ motivations, ability, socio-economic status, 
preparedness, and other external factors (Tinto; 1993; Bean & Eaton, 2000; Braxton & 
Hirschy, 2005;Jones et ah, 2008; Hawkins & Larabee, 2009). 

Particularly, for first-year academic achievement at the university, the level of academic 
readiness or preparedness is of critical importance. Literature suggests that a reason for 
the poor performance of the higher education sector is that students enter the system 
underprepared (Scott,Yeld & Hendry, 2007; Strydom as cited in Joubert, 2002). Academic 
readiness is broadly defined as the level of preparation a student needs in order to enrol 
and succeed, without remediation, in a credit-bearing programme at a higher education 
institution (Conley, 2007). More specifically, preparedness refers to being prepared in 
reading, writing and mathematical skills (Van Dyk & Weideman, 2004; Cliff, 2014). 

The readiness model of Conley (2007) shows that readiness for university education 
is not only associated with academic performance at school or with measures of ability on 
psychometric tests, but also with socio-cultural and motivational factors. The participants 
of Byrd and MacDonald’s study, for instance, identified the following additional factors 
associated with readiness, namely: skills in time-management; motivational factors; 
background factors; and student self-concept (Byrd & MacDonald, 2005). Other researchers 
pay attention to the non-cognitive and/or demographic characteristics of students as 
influencers of readiness for university (Sedlacek, 2004, 2005; Camara, 2005a, 2005b). 

The four categories of readiness that were identified by Byrd and MacDonald’s (2005) 
qualitative study are confirmed by Conley s (2007) research on readiness for university over 
a number of years. Conley further suggests a broad definition of readiness that includes 
cognitive strategies, acquiring content knowledge, academic behaviours, and contextual 
knowledge and skills. Conley (2007) explains that the various elements of readiness are 
neither mutually exclusive nor perfectly nested because they interact with and affect one 
another extensively. Entry characteristics in the form of demographic variables have been 
shown to predict accomplishment later in one’s academic career (Sedlacek, 2005). 
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Keup (2008) names four issues that impact on student readiness: the shift to a truly 
multicultural student body; mental and emotional healthcare needs; a utilitarian view of 
higher education; and an integration of new technologies. In considering the first-year 
experience, Kift (2009) says that the challenge of moving from research and theory into 
practice is becoming more difficult. This makes it even more important to determine 
whether our practices will have any impact on student success. The interventions provided 
by the FSAs can be classified as informal learning opportunities (Wawrzynski and Baldwin, 
2014) that also forge links with the formal ones taking place in the classroom. While 
interventions by the FSAs may not fall neatly into Kuh’s (2008) high-impact practices, they 
are seen as an attempt to address some of the needs of first-time-entering students. 

Early alert and referral strategy 

Our institution adopted an early alert and referral system early in 2010 to facilitate the 
transition from school to university in order to have students fully integrated into the 
university environment within the first quarter of the first year. Among the activities were 
academic introductions to the disciplines before the start of the official year, allocation of 
mentors to particular students, monitoring student achievement after the first test (in a 
particular faculty), and the placement of advisors to provide support within each faculty. 

In line with literature on student readiness (Van der Merwe & Pina, 2008; Van Zyl, 
Gravett & De Bruin, 2012; Rassen et al, 2013; Van Zyl, 2013), students are assessed on their 
academic readiness. Our Student Academic Readiness Survey (STARS) was developed 
to function — in conjunction with demographic variables, high-school marks (Admission 
Point Score) and the National Benchmark Test — as early warning indicators of failure 
or dropout among first-year students. The STARS is a low-stakes, self-report survey 
measuring non-cognitive variables on 115 items, administered during the orientation 
week. This survey has been administered, since 2010, to over 42 000 students. It is a norm- 
referenced test and consists of 26 non-cognitive dimensions. The objectives of the STARS 
are to act as early-warning indicators of failure or dropout among first-year students and to 
categorise students into groups for specific interventions. 

Students identified through the STARS are referred to a peer mentorship programme 
for transitional support and/or an FSA for academic support and advice. The FSA refers 
students with financial and accommodation challenges to the relevant sections. This 
study will focus on the academic development interventions supplied by the FSAs. The 
responsibilities of the FSAs are to make contact with students and invite them to an 
intervention programme; advise such students about reducing their risks; provide study 
skills and time management workshops; monitor particular students’ results as an early alert 
of their progress; provide support to self-referred students; and assist students requiring 
advice about programme changes. 

The FSAs are also required to keep records of the students who attend their 
intervention programmes. The data about the number of students identified for the 
intervention programmes as well as the students’ participation rates are necessary for the 
programme to have value during the first semester of the first year. Research suggests that 
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not only student support services, but the system as a whole, influence students’ learning 
experiences and success (Tinto, 2013). While efforts are also being made to encourage 
lecturers to support students within their discipline, this is not the focus of this article. Here, 
we focus on the interventions made by the FSAs. 

Methodology 

The current study focuses on the quantitative analysis of survey data in combination with 
high-school academic results, student demographic data and results from the National 
Benchmark Test (NBT). The aforementioned data are used in predictive analytics to 
identify students for various intervention programmes on campus. 

Data collection method or procedure 

The STARS is administered to students attending the orientation week at the beginning 
of each academic year. The survey is intended for all first-time-entering, first-year students. 
However, not all new students are able to attend the orientation week and, in some cases, 
returning and transferring students also attend it. The surveys are administered in paper- 
and-pencil format and electronically. The results of the STARS feed directly into the 
institutions business intelligence software, called the STARS Student Retention System. 
Each students STARS profile is programmatically compared with his or her high school 
academic results, student demographic data and results from the NBT. Predictive analytics 
of the data over a number of years were used to develop academic risk profiles of students. 
The STARS retention system uses the algorithms of the predictive analytics to identify the 
students who may be at risk. The system produces automatic reports that are used by the 
FSAs to contact students about a variety of intervention programmes. 

The criteria for selection for academic advising, which are evaluated in this study, 
include high school academic performance in relation to the admission requirements per 
programme. As part of this study we have included the NBT as an additional criterion for 
selecting at-risk students because research shows that the three NBT sub-tests generally act 
as contributing signals, with National Senior Certificate (NSC) results, in explaining first- 
year academic achievement (Lemmens, 2013).The results of the STARS are mainly used as 
qualitative information for tailoring the intervention programme. 

FSAs are required to keep records of the students who attend academic advising and 
the number of sessions they attend. The number of sessions ranged from zero to nine 
sessions.These sessions were clustered into three, roughly even, groups in order to perform 
a Pearson’s Chi-square analysis from contingency tables (Field, 2005). The students who 
attended zero sessions were contacted but chose not to attend any of the sessions. 

Three distinct groups — at-risk, borderline and successful — were used by the FSAs for 
further academic development interventions in the second semester. The data for this is not 
presented here as the focus is on the first semester only. 

The research question for this study is: Does the number of academic advising sessions 
decrease the academic risk of students who were predicted to be at risk upon entry to the 
institution? 
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Data analysis 

The data analysis for this study can be categorised as descriptive analysis of the demographic 
data and institutional sample data. Inferential statistics, namely a Pearson s Chi-square, was 
used in combination with contingency tables because the data were clustered into discrete 
categories. A Chi-square analysis was also used to test a hypothesis with an associated 
significance indication (Field, 2005). 

In order to evaluate the academic success of students, a cluster analysis was performed 
to identify students who had performed academically poorly in the first semester. Cluster 
analysis is a statistical method for finding relatively homogeneous clusters of cases based 
on measured characteristics. The k-means algorithm clustering method was utilised to 
analyse the data (Field, 2005). The variables that were used in the first phase of the analysis, 
performed by our Institutional Planning section, were the ratio of credits registered 
versus credits failed; the average mark for the first semester; the ratio of modules in which 
students performed poorly; and the high school English mark. In the second round 
of the analysis, the borderline students were extracted from the data set and split into 
Sciences and non-Sciences groups. The k-means cluster analysis was performed on these 
two groups separately. The variables that were used in the second phase were the ratio of 
credits registered versus credits failed; the average mark for the first semester; and the ratio 
of modules in which students performed poorly. However, the high school English mark 
was removed. This allowed identification of borderline students at the granular level, thus 
splitting the lower borderline from the true borderline and upper borderline students. The 
lower borderline students became part of the at-risk group. 

The following outcome assessment models will be used to evaluate the outcomes as 
they are presented in the evaluation framework: 

Model 1: Outcomes assessment of at-risk students 


Characteristics of at-risk students at entry 

Academic development 

Transition 

Financial and 

accommodation 


I 


Participation rates 


Successfully complete intervention Discontinue intervention 


j 


Academic outcomes 

Fail first semester 

Dropout/course change 

Academic risk cluster 
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In this model, the students who were identified as being at risk with predictive analytics 
were referred to one of the three support services mentioned above. They are subsequently 
prearranged in two groups, namely students who successfully completed the intervention 
and students who discontinued or defaulted on the intervention. The two groups will be 
compared in relation to their first-semester academic achievement, their academic risk 
cluster and dropout rate.The purpose is to determine whether students who made use of the 
intervention services were more likely to be successful than the students who defaulted on 
the intervention. In this study the focus is on the cluster analysis of the academic outcomes 
in relation to participation in academic development sessions facilitated by the FSAs. 

Sample 

The population for this study consists of 12 916 students enrolled in their first year in 2013. 
First-time-entering, first-year students numbered 8 515, with the remainder of the students 
being both returning students (students who did not successfully progress to the second 
year of study) and transferring students (students who changed course or transferred from 
other institutions). 

A total of 7 033 students completed the STARS and, from the initial risk criteria, 966 
students were identified as being at risk. After applying additional risk criteria explained in 
the data collection procedure above, 200 students were identified as being at risk. The FSAs 
followed these students up. The FSAs kept records of the 200 students for the purpose of 
this study. 

Coyne (1997) refers to all sampling as being purposeful. He turns to Pattons view 
(1990, p. 69), that “qualitative inquiry typically focuses in depth on relatively small samples, 
even single cases, selected purposefully”. The purpose of choosing the 200 students is that 
they interacted with the FSAs in terms of being provided with a form of intervention. 

Ethical considerations 

Confidentiality of first-year students was maintained at all costs. Students were informed of 
the purpose of the survey prior to its administration. Students had to log in to the student 
portal with their student numbers and passwords to gain access to their individual reports. 
Only reports of students who agreed to make their information accessible to university staff 
were available by proxy access to FSAs, counselling staff and deans of faculty. Students were 
advised to make use of support structures, but not forced to do so. These students were also 
briefed about the method through which they had been identified and what the support 
programme entailed. Students who decided not to make use of recommended services 
either did not attend the sessions or indicated their decision to default verbally to the FSA 
when invited to the intervention programme. 

Results 

Our results focus on the descriptive statistics of the sample and the quantitative evaluation 
of the intervention programme facilitated by the FSAs. 
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Sample 

The distribution of first-year students by faculty and by admittance type can be observed 
in Table 1. 


Table 1: Biographical data of first-year students by faculty and admission type 


Faculty 

New 

Returning 

Transfer 

Total 

Humanities 

1 394 

244 

80 

1 718 

Natural and Agricultural 
Sciences 

1 716 

663 

81 

2 460 

Law 

173 

180 

9 

362 

Theology 

56 

11 

7 

74 

Economic and 
Management Sciences 

1 717 

383 

95 

2 195 

Veterinary Science 

131 

42 

27 

200 

Education 

914 

98 

54 

1 066 

Health Sciences 

569 

646 

651 

1 866 

Engineering, Built 
Environment and IT 

1 845 

1 085 

45 

2 975 

Total 

8 515 

3 352 

1 049 

12 916 


66% 

26% 

8% 

100% 


From the population of first-year students admitted to the university (12 916), a total of 
7 033 students completed the STARS. Of the 7 033 students completing the STARS 
during the orientation programme, 93% are first-time-entering, first-year students, 2% 
of the registered students are students returning to the first year and 4% of the registered 
students have transferred from other institutions. One per cent of the students are labelled as 
“unknown” on their student record. The target audience for the student academic readiness 
survey was first-time-entering, first-year students and, to a large extent, this was achieved. 

The distribution in participation rates in the STARS by faculty is equivalent to the 
distribution of the student population as presented in Table 2, thus indicating that the 
students who completed the STARS are a representative sample of the first-time-entering, 
first-year student population. 

From Table 3 it is evident that the majority of students who completed the STARS 
are female, which is in accordance with the gender profile of the undergraduate student 
population (female = 58.5% and male = 41.5%) and first-year students in particular (female 
= 58.2% and male = 41.8%) at our own institution. 
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Table 2: Distribution of students completing the STARS by faculty 


Faculty 

Frequency 

Percentage 

Economic and Management Sciences 

1 374 

19.5 

Education 

834 

11.9 

Engineering, Built Environment and 
IT 

1 550 

22.0 

Health Sciences 

469 

6.7 

Humanities 

986 

14.0 

Law 

164 

2.3 

Natural and Agricultural Sciences 

1 500 

21.3 

Theology 

36 

0.5 

Veterinary Science 

120 

1.7 

Total 

7 033 

100.0 


Table 3: Distribution of students completing the STARS by gender 


Gender 

Frequency 

Percentage 

Female 

4 120 

58.6 

Male 

2 913 

41.4 

Total 

7 033 

100.0 


Table 4: Distribution of students completing the STARS by race 


Racial group 

Frequency 

Percentage 

Asian 

363 

5.2 

Black 

2 705 

38.5 

Coloured 

178 

2.5 

Other 

34 

0.5 

Unknown 

6 

0.1 

White 

3 747 

53.3 

Total 

7 033 

100.0 


From Table 4 it is evident that the majority of students who completed the STARS 
are categorised as white, which is roughly in accordance with the racial profile of the 
undergraduate student population and first-year students in particular. Black students make 
up 38.5% of the sample, while the proportion of black students in the population of first- 
time-entering, first-year students is 41.1%. The proportion of white students completing 
the STARS is 53.3%, while the population of white, first-time-entering, first-year students 
is 50.7%. Black students are thus slightly under-represented in the sample and white 
students are slightly over-represented in the sample. 
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Academic proficiency of students completing the STARS 

The academic proficiency of students who completed the STARS is presented in Table 5. 
Academic proficiency, in this instance, is measured by the Admission Point Score (APS) as 
well as the subtests of the National Benchmark Test (NBT). These scores show, to some 
extent, the academic skills and/or knowledge that a student has acquired up to a certain 
point. Universities can use this information to set benchmarks for the level of knowledge 
and/or skills that students require to have a fair chance of being successful at a particular 
programme. Thus, they not only say something about the students’ current ability, but also 
predict academic outcomes in the future based on what is known presently. 


Table 5: Average high-school academic performance on the APS and NBT 



N 

Min 

Max 

Mean 

Std deviation 

APS 

6 

574 

15 

50 

34.50 

4.589 

NBT-AL 

5 

139 

28 

92 

62.64 

11.311 

NBT-QL 

5 

139 

13 

96 

58.88 

15.128 

NBT-Math 

4 

510 

19 

97 

48.84 

15.947 

Sample total 

4 

276 






From Table 5 the mean APS score was 34.5. On the other hand, the mean score for 
the NBT—Academic literacy subtest was 62.64, while the mean score for the NBT— 
Quantitative literacy subtest was 58.88 and the mean score for the NBT—Mathematics 
subtest was 48.84. The sample total for this table is 4 276 because not all students have to 
complete the NBT results, and the sample total represents the number of students who 
have scores for all four of the variables. 

First-semester students ‘at risk’ according to the STARS 

Table 6 shows that 966 students were identified as being at risk by the STARS Student 
Retention System at the beginning of the academic year, using only APS as the criteria. 
For the purpose of this study, the NBT was included in the selection criteria to identify 
students who are at a greater risk. A total of 200 students were selected for this purpose. 

Internal research at our institution on NSC subjects and APSs with the STARS results 
shows that the NSC — and, more specifically, the APS — only partly explains academic 
outcomes (Lemmens, 2013). When other variables, such as NBT results and psychosocial 
variables, are added, they can increase the accuracy of the prediction model. 

From Table 7 it is evident that students who were selected based on APS criteria only 
have higher mean scores on the NBT-AL, NBT-QL and NBT-Math than the students 
selected with APS and NBT criteria (60.64 vs 56.23; 57.24 vs 49.91; 45.27 vs 39.05 
respectively). One can see that the APS scores of both groups were almost equal and that 
the addition of the NBT subtest provides additional information to help with the accuracy 
of the prediction model.The students who were contacted were prioritised because of their 
performance on the NBT and would possibly be at greater academic risk. 
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Table 6: Prediction criteria applied 


Faculty 

Criteria: APS 

Criteria: APS and 

NBT 

Economic and Management Sciences 

264 

42 

Education 

97 

23 

Engineering, Built Environment and IT 

215 

13 

Health 

33 

2 

Humanities 

242 

43 

Law 

12 

3 

Natural and Agricultural Studies 

83 

65 

Theology 

7 

0 

Veterinary Science 

13 

9 

Grand total 

966 

200 


Table 7: Descriptive statistics of participation in FSA session/s and proficiency scores 


APS criteria only 
(N = 766) 

N 

Min 

Max 

Mean 

Std 

deviation 

APS 

763 

17 

38 

29.83 

3.679 

NBT-AL 

519 

29 

85 

60.64 

11.319 

NBT-QL 

519 

20 

91 

57.24 

15.981 

NBT-Math 

439 

20 

88 

45.27 

14.396 

APS and NBT 
criteria (N = 200) 

N 

Min 

Max 

Mean 

Std 

deviation 

APS 

196 

3 

37 

29.09 

3.523 

NBT-AL 

125 

31 

83 

56.23 

12.363 

NBT-QL 

125 

19 

84 

49.91 

14.011 

NBT-Math 

112 

22 

71 

39.05 

11.939 


Academic development outcomes assessment 

In this section, the students who were identified as being at risk based on academic criteria 
will be assessed against the academic risk cluster as the outcome variable. The students 
who were identified and referred to the FSAs will be divided into three groups, analysed 
according to the number of sessions in which they participated (grouped for analysis 
purposes). 
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Table 8: Cross-tabulation of academic risk cluster and number of FSA sessions 
participated in 



The two variables in the contingency table (cross-tabulation) in Table 8 — academic risk 
cluster and the number of FSA sessions participated in — are significantly associated on the 
Pearsons Chi-square statistic (13.60, df(4), p = 0.009).This implies that there is a significant 
relationship between the risk cluster in which a student is observed and the number of 
academic advising sessions in which a student participated with the FSAs.The category of 
“Zero sessions” refers to the students who were invited to the interventions but did not 
participate (n = 60). The number of students attending one session were 84 and number of 
students attending two or more sessions were 56. The results show that 41.8% of the at-risk 
students fall into the zero session, 40.3% participated in one session and 17.9% participated 
in two or more sessions. The students classified as borderline with the cluster analysis had 
slightly larger numbers of students attending one session. Namely, 38.7% of the students 
attended one session, 32% of the borderline students attend zero sessions and 29.3% attended 
two or more sessions. In contrast, 13.8% of the successful students attended zero sessions, 
48.3% attended one session and 37.9% of the students attended two or more sessions. 

Based on the cluster analysis percentages, one could conclude that the optimal number 
of sessions that at-risk students should attend, to move from being classified as being at 
risk upon entry into the institution to being classified as successful, is one. One can also 
observe a larger number of students within the “Successful” category. However, students 
attending one session were in the majority, which will influence the relative frequencies 
and associated percentages. In order to accommodate the difference in total frequencies 
of students attending interventions, the relative percentages of the number of sessions in 
relation to the academic cluster has to be investigated. 

The results of the calculation of the frequencies and column totals in Table 8 
(percentages not shown here but explained in this paragraph) show that successful students 
were more likely to have attended two or more sessions (39.3%) than students who 
attended one session (33.3%) or no sessions at all (13.3%). Conversely, the at-risk students 
tended not to attend any sessions (46.7%) compared with students who participated in 
one session (32.1%) and students who participated in two or more sessions (21.4%). The 
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students classified as borderline could have attended zero (40%), one (34.5%), or two or 
more (39.3%) sessions without showing a clear trend in the contribution of attending more 
sessions to risk movement. Generally, the results thus show that of the high-risk group 
contacted by the FSAs, students who attend more sessions with the advisor are more likely 
to have successful first-semester academic outcomes. Furthermore, there is evidence to 
show that academic advice is an effective intervention for high-risk students in the short 
term. However, this finding needs to be tested further. Further research will be undertaken 
on students who did not receive any intervention, but who were identified by the STARS, 
to determine if they could have been more successful had they received academic advice. 
Such a comparison could have worked well with the students whom the STARS system 
identified as not being at risk. 

Conclusion 

The research from this study found that 766 STARS students who were selected using APS 
criteria and who did not attend FSA interventions were less likely to be at risk than the 
200 who were selected based on the APS and NBT results participated. Adding the subtest 
of the NBT to the identification of students for the academic intervention programmes 
has improved the ability to predict the success of students entering the university, allowing 
the university to be more active in selecting at-risk students with predictive analytics 
and recommending interventions proactively. An outcomes assessment of this analysis- 
intervention—evaluation framework has shown promising results for the implementation of 
academic advising in this case, and has allowed us to improve the intervention programme 
as well as the monitoring of at-risk students. 

The results show that only 32 out of200 students were able to move out of academic risk 
without attending any academic advising — however, most of these students were at borderline 
academic achievement.This could be attributed to students making changes to their academic 
behaviour due to the knowledge that they were being observed (the Hawthorne effect) or 
because they consulted elsewhere. The results show that 28 of the 60 students who did not 
attend any sessions are from the at-risk category (46.7%). It is also evident that students who 
attend only one session have a one in three chance of being either at risk, borderline or 
successful. This indicates that attending only one session does not seem to be clearly beneficial 
to this group of students. Evidently, the tipping point in the effectiveness of academic advising 
is student participation in two or more sessions. Students who attend two or more sessions are 
likely to be successful or, at least, to become borderline academic achievers. Clearly, students 
who do not use FSA services are at a higher risk. Thus, active participation in intervention 
programmes is of value to academically underprepared students. 

Through the STARS and the employment of FSAs, the institution has shown its 
commitment to interventions for improving student success. It is taken further within 
the faculties where students identified as being at risk are supported by tutors and other 
academic practices. 

The findings have also shown the value of using NBT results as additional criteria 
for identifying at-risk students for academic advice programmes. Research has used NBT 
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mostly in comparative analyses of NSC marks and NBTs for placement purposes (Fleisch, 
Schoer & Cliff, in press). 
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